Reproducing apparatus, reproducing method, and recording medium

ABSTRACT

A reproducing apparatus simultaneously reproduces a first video signal and a second video signal reproduced at any timing against the first video signal. A first (second) converting section outputs the first (second) video signal having a first frame frequency alternately for three successive frames and two successive frames to convert the first (second) video signal having the first frame frequency into the video signal having a second frame frequency. They have a relationship of 2 to 5. A controlling section performs control such that timing at which a field group of the three successive frames of the first video signal having the second frame signal is changed to a field group of the two successive frames thereof matches timing at which a field group of the three successive frames of the second video signal having the second frame signal is changed to a field group of the two successive frames thereof.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2007-323294 filed in the Japanese Patent Office on Dec.14, 2007, the entire contents of which being incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a reproducing apparatus, a reproducingmethod, and a recording medium that can be suitably used when aplurality of video signals are simultaneously reproduced.

2. Description of the Related Art

In recent years, as the digital high definition television broadcastshave been practically implemented, in the National Television SystemCommittee (NTSC) video data format, progressively scanned video datahaving a frame frequency of 59.94 Hz (60 Hz), which are capable ofdisplaying more precise pictures, are being dominantly used instead oftraditionally interlaced scanned video data having a frame frequency of29.97 Hz (30 Hz).

In such an environment, display apparatus such as flat panel displayunits using a liquid crystal display (LCD) or a plasma display panel(PDP) that can display progressively scanned video data having a framefrequency of 59.94 Hz (60 Hz) are being placed in the market. Displayapparatus that the users use as mainstream units, however, are thosethat can display interlaced scanned video data having a frame frequencyof 29.97 Hz (30 Hz).

On the other hand, movie pictures are produced using traditional opticalfilms as progressively scanned video data with 24 frames per second,namely having a frame frequency of 24 Hz.

Thus, when progressively scanned video data having a frame frequency of24 Hz are displayed by a display apparatus that can displayprogressively scanned video data having a frame frequency 29.97 Hz (30Hz), it is necessary to cause the reproducing apparatus side to convertprogressively scanned video data having a frame frequency of 24 Hz intointerlaced scanned video data having a frame frequency of 30 Hz (fieldfrequency of 60 Hz).

When progressively scanned video data having a frame frequency of 24 Hz(hereinafter referred to as 24p video data) are converted intointerlaced scanned video data having a frequency of 30 Hz (fieldfrequency of 60 Hz) (hereinafter referred to as 60i video data), 24pvideo data are alternately converted into three frames and two frames of60i video data according to a predetermined conversion pattern. Thus,this process is referred to as the 3-2 pull-down (or 2-3 pull-down)process.

Next, the 3-2 pull-down process, which converts 24p video data into 60ivideo data, will be described. There is a relationship of 2 to 5 inframe frequencies between 24p video data and 60i video data. Thus, asshown in FIGS. 1A and 1B, in the 3-2 pull-down process, for example,whenever each frame of 24p video data is output, three successive framesand two successive frames of 60i video data (see FIG. 1B) arealternately output. Thus, in the 3-2 pull-down process, 24p video dataare converted into 60i video data.

International Publication No. 04/032494 (hereinafter referred to asPatent Document 1) describes a technique that performs the 3-2 pull-downprocess that converts 24p video data based on a film material into 60ivideo data.

SUMMARY OF THE INVENTION

Some recent reproducing apparatus have a so-called picture-in-picturefunction that reproduces a first picture as a main picture, reproduces asecond picture as a sub picture at any timing, and superimposes the twopictures so as to simultaneously reproduce them.

When such a reproducing apparatus converts 24p video data into 60i videodata by the 3-2 pull-down process and reproduces the 60i video data, ifthe apparatus simultaneously reproduces the two pictures using thepicture-in-picture function, the second picture may appear in a combshape depending on timing at which the apparatus reproduces the secondpicture, namely so-called combing occurs.

In view of the foregoing, it would be desirable to provide a reproducingapparatus, a reproducing method, and a reproducing program that canprevent combing from occurring when the picture-in-picture function isused.

According to an embodiment of the present invention, there is provided areproducing apparatus which simultaneously reproduces a first videosignal and a second video signal reproduced at any timing against thefirst video signal. The reproducing apparatus includes a firstconverting section, a second converting section, and a controllingsection. The first converting section outputs the first video signalhaving a first frame frequency alternately for three successive framesand two successive frames so as to convert the first video signal havingthe first frame frequency into the first video signal having a secondframe frequency, the first frame frequency and the second framefrequency having a relationship of 2 to 5. The second converting sectionoutputs the second video signal having the first frame frequencyalternately for three successive frames and two successive frames so asto convert the second video signal having the first frame frequency intothe second video signal having the second frame frequency, the firstframe frequency and the second frame frequency having a relationship of2 to 5. The controlling section performs control such that timing atwhich a field group based on fields of the three successive frames ofthe first video signal having the second frame signal is changed to afield group based on fields of the two successive frames thereof matchestiming at which a field group based on fields of the three successiveframes of the second video signal having the second frame signal ischanged to a field group based on fields of the two successive framesthereof.

According to an embodiment of the present invention, there is provided areproducing method of simultaneously reproducing a first video signaland a second video signal reproduced at any timing against the firstvideo signal. The first video signal having a first frame frequency isoutput alternately for three successive frames and two successive framesso as to convert the first video signal having the first frame frequencyinto the first video signal having a second frame frequency, the firstframe frequency and the second frame frequency having a relationship of2 to 5. The second video signal having the first frame frequency isoutput alternately for three successive frames and two successive framesso as to convert the second video signal having the first framefrequency into the second video signal having the second frame, thefirst frame frequency and the second frame frequency having arelationship of 2 to 5. Control is performed such that timing at which afield group based on fields of the three successive frames of the firstvideo signal having the second frame signal is changed to a field groupbased on fields of the two successive frames thereof matches timing atwhich a field group based on fields of the three successive frames ofthe second video signal having the second frame signal is changed to afield group based on fields of the two successive frames thereof.

According to an embodiment of the present invention, there is provided arecording medium storing a reproducing program which causes a computerapparatus to execute a reproducing method of simultaneously reproducinga first video signal and a second video signal reproduced at any timingagainst the first video signal. The first video signal having a firstframe frequency is output alternately for three successive frames andtwo successive frames so as to convert the first video signal having thefirst frame frequency into the first video signal having a second framefrequency, the first frame frequency and the second frame frequencyhaving a relationship of 2 to 5. The second video signal having thefirst frame frequency is output alternately for three successive framesand two successive frames so as to convert the second video signalhaving the first frame frequency into the second video signal having thesecond frame, the first frame frequency and the second frame frequencyhaving a relationship of 2 to 5. Control is performed such that timingat which a field group based on fields of the three successive frames ofthe first video signal having the second frame signal is changed to afield group based on fields of the two successive frames thereof matchestiming at which a field group based on fields of the three successiveframes of the second video signal having the second frame signal ischanged to a field group based on fields of the two successive framesthereof.

According to an embodiment of the present invention, changes of fieldgroups of primary video data as a first video signal are matched withthose of secondary video data as second video signal, combing thatoccurs when the secondary video data is reproduced can be prevented.

These and other objects, features and advantages of the presentinvention will become more apparent in light of the following detaileddescription of a best mode embodiment thereof, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A and FIG. 1B are schematic diagrams describing a 3-2 pull-downprocess;

FIG. 2 is a schematic diagram showing an outline of a BD-ROM data model;

FIG. 3 is a schematic diagram describing an index table;

FIG. 4 is an UML diagram showing a relationship of clip AV streams, clipinformation, clips, play items, and play lists;

FIG. 5 is a schematic diagram describing a method of referring to thesame clip from a plurality of play lists;

FIG. 6 is a schematic diagram describing a sub path;

FIG. 7 is a schematic diagram describing the reproduction of video datawhen a picture-in-picture function is used;

FIG. 8 is a flow chart describing a reproducing process when thepicture-in-picture function is used;

FIG. 9 is another flow chart describing the reproducing process when thepicture-in-picture function is used;

FIG. 10A and FIG. 10B are schematic diagrams describing occurrence ofcombing;

FIG. 11 is a schematic diagram describing a method of preventing combingfrom occurring according to an embodiment of the present invention;

FIG. 12 is a block diagram showing a structure exemplifying areproducing apparatus according to this embodiment of the presentinvention;

FIG. 13A and FIG. 13B are block diagrams showing structures exemplifyinga primary video converting section and a secondary video convertingsection;

FIG. 14 is a schematic diagram describing combinations of encodingformats of primary video data and secondary video data;

FIG. 15 is a schematic diagram describing combinations of encodingformats of primary video data and secondary video data;

FIG. 16 is a flow chart describing a process that calculatesreproduction start timing of secondary video data according to thisembodiment of the present invention;

FIG. 17 is a schematic diagram describing the reproducing processaccording to this embodiment of the present invention;

FIG. 18 is a flow chart describing a calculating process for valuenative_vsync_count representing the original number of Vsync's forpictures that have been displayed;

FIG. 19 is a flow chart describing a calculating process for valueprsn_vsync_cnt representing the number of Vsync's for pictures that havebeen displayed;

FIG. 20 is a schematic diagram describing a reproducing method accordingto a first modification of this embodiment of the present invention;

FIG. 21 is a flow chart describing a calculating process forreproduction start timing of secondary video data according to the firstmodification of this embodiment of the present invention;

FIG. 22 is a schematic diagram describing the reproducing processaccording to the first modification of this embodiment of the presentinvention;

FIG. 23 is a schematic diagram describing a reproducing method accordingto a second modification of this embodiment of the present invention;

FIG. 24 is a flow chart describing a calculating process forreproduction start timing of secondary video data according to thesecond modification of this embodiment of the present invention;

FIG. 25 is a schematic diagram describing a reproducing processaccording to the second modification of this embodiment of the presentinvention; and

FIG. 26 is a block diagram showing a structure exemplifying anotherreproducing apparatus according to this embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Next, with reference to the accompanying drawings, embodiments of thepresent invention will be described. According to the embodiments of thepresent invention, when a reproducing apparatus reproduces audio/video(AV) data are reproduced from a record medium such as a Blu-ray discread-only memory (BD-ROM) (Blu-ray is a registered trademark), combingthat occurs when a plurality of video data are reproduced using thepicture-in-picture function is prevented. In the following description,it is assumed that AV data are reproduced from a BD-ROM.

For easy understanding for the embodiments of the present invention, themanagement structure of AV data recorded on a BD-ROM, which is aread-only Blu-ray disc specified in “Blu-ray Disc Read-Only Format Ver1.0 part 3 Audio Visual Specifications” will be described in brief.Hereinafter, the management structure of this BD-ROM is referred to asthe Blu-ray Disc Movie & Video (BDMV) format.

A bit stream encoded according to an encoding system such as MovingPictures Experts Group (MPEG) video or MPEG audio and multiplexedaccording to the MPEG2 System is referred to as a clip AV stream (or AVstream). A clip AV stream is recorded as a file on the disc by a filesystem defined in “Blu-ray Disc Read-Only Format Part 2”, which is oneof “Blue-ray Disc” standards. This file is referred to as a clip AVstream file (or AV stream file).

A clip AV stream file is a management unit on the file system. Thus, aclip AV stream file may not be a management system that the user caneasily understand. Taking into account of user's convenience, it isnecessary to provide a mechanism that combines and reproduces videocontent divided into a plurality of clip AV stream files, a mechanismthat reproduces a part of a clip AV stream file, and a mechanism thatrecords information necessary for smoothly performing a specialreproducing operation and a cue-reproducing operation as a database to adisc. “Blu-ray Disc Read-Only Format part 3”, which is one of Blu-rayDisc standards, specifies this database.

FIG. 2 shows an outline of a BD-ROM data model. As shown in FIG. 2, aBD-ROM has a data structure composed of four layers. The lowest layer isa layer for clip AV streams (this layer is conveniently referred to asthe clip layer). The immediately upper layer is a layer for movie playlists (Movie PlayList) and play items (PlayItem) that representreproduction positions of clip AV streams (this layer is convenientlyreferred to as the play list layer). The further immediately upper layeris a layer for movie objects (Movie Object) and so forth. Each movieobject is composed of a command that represents a reproduction order ofmovie play lists (this layer is conveniently referred to as the objectlayer). The highest layer is a layer for an index table that managestitles and so forth recorded on the BD-ROM (this layer is convenientlyreferred to as the index layer).

Next, the clip layer will be described. A clip AV stream is a bit streamof which video data and audio data have been multiplexed in the format,for example, of an MPEG2 transport stream (MPEG2 TS). Informationregarding the clip AV stream is recorded as clip information in a file.

In addition, streams for a subtitle and a menu displayed in associationwith content data of video data and audio data are multiplexed in theclip AV stream. A graphics stream for a sub title is referred to as apresentation graphics (PG) stream. On the other hand, a stream for dataof a menu is referred to as an interactive graphics (IG) stream. Aninteractive graphics (IG) menu reproduced from an interactive graphics(IG) stream can contain items, for example, of buttons, icons,thumbnails, or the like that the user can input to perform predeterminedoperations.

A clip AV stream file and a clip information file for corresponding clipinformation are treated as one object and referred to as a clip. Inother words, a clip is one object composed of a clip AV stream and clipinformation.

A file is generally treated as a byte sequence. Content of a clip AVstream file is arranged on the time base. Entry points of clips arerepresented mainly on the basis of times. When a time stamp of an accesspoint to a predetermined clip is represented, address information atwhich data are read from the clip AV stream file can be used.

Next, the play list layer will be described. A movie play listrepresents an AV stream file to be reproduced and is composed of sets ofreproduction start points (IN points) and reproduction stop points (OUTpoints) that represent regions to be reproduced of the represented AVstream file. A set of information of the reproduction start point andthe reproduction stop point is referred to as a play item (PlayItem). Amovie play list is composed of a set of play items. When a play item isreproduced, a part of an AV stream file to which the play item refers isreproduced. In other words, a region of a clip is reproduced based oninformation of an IN point and an OUT point of a play item.

Next, the object layer will be described. A movie object contains anHDMV navigation command program (HDMV program) and a terminalinformation associated with the movie object. An HDMV program is acommand that controls the reproduction of a play list. Terminalinformation contains information that permits the user to perform aninteractive operation for the BD-ROM player. User operations such ascalling of a menu screen and title search are controlled based on theterminal information.

A BD-J object is composed of an object written in a Java program(registered trademark). Since a BD-J object is beyond scope of thisembodiment of the present invention, its detailed description will beomitted.

Next, the index layer will be described. The index layer is composed ofan index table. The index table is a top table that defines titles ofthe BD-ROM disc. The reproduction of titles of the BD-ROM disc iscontrolled by a module manager of BD-ROM resident system software basedon title information stored in the index table.

In other words, as outlined in FIG. 3, any entries in the index tableare referred to as titles. A first playback (First Playback), a top menu(Top Menu), and titles (Titles) #1, #2, . . . and so forth as areentries of the index table and titles. Each title represents a link to amovie object or a BD-J object. Each title represents either an HDMVtitle or a BD-J title.

If content recorded on the BD-ROM is a movie, the first playback is acommercial movie (trailer) of a movie company and is displayed beforethe main movie. If content is a movie, the top menu is a menu screen onwhich the user can select the reproduction of the main movie, search fora chapter, set a subtitle and its language, and select the reproductionof a bonus movie and so forth. In addition, the titles may be moviesthat the user can select from the top menu. The titles may be menuscreens.

FIG. 4 is a unified modeling language (UML) diagram showing arelationship of the foregoing clip AV streams, clip information (StreamAttributes), clips, play items, and play lists. One play list iscorrelated with one or a plurality of play items. One play item iscorrelated with one clip. One clip can be correlated with a plurality ofplay items that differ in start point and/or stop point. One clip refersto one clip AV stream file. Likewise, one clip refers to one clipinformation file. One clip AV stream file and one clip information filehave a relationship of 1 to 1. When such a structure is defined, anondestructive reproduction order can be represented to reproduce adesired region without necessity of changing the clip AV stream file.

In addition, as shown in FIG. 5, a plurality of play lists can refer tothe same clip. In addition, a plurality of clips can be represented fromone play list. An IN point and an OUT point of a play item of a playlist can refer to a clip. In the example shown in FIG. 5, a play item320 of a play list 310 refers to a clip 300. A play item 321 of a playlist 311 that is composed of play items 321 and 322 refers to a regionrepresented by an IN point and an OUT point. A play item 322 of a playlist 311 refers to a clip 301 for a region represented by an IN pointand an OUT point of a play item 322 and for a region represented by anIN point and an OUT point of a play item 323 of a play list 312 composedof play items 323 and 324.

As exemplified in FIG. 6, a play list can have a sub path correspondingto a sub play item against a main path corresponding to a play itemmainly reproduced (main play item). The sub play item can be correlatedwith a plurality of clips that differ from each other. The sub play itemcan selectively refer to one of a plurality of clips correlatedtherewith. Although details will be omitted, only when a play listsatisfies a predetermined condition, the play list can have a sub playitem.

A video stream contained in a clip to which a main play item thatcomposes the main path (hereinafter referred to as the primary videostream) refers can be reproduced simultaneously in combination withanother stream. Other streams reproduced in combination with the primaryvideo stream include a video stream contained in a clip to which a subplay item that composes the sub path corresponding to the main path(hereinafter referred to as the secondary video stream) refers. The subpath is used to represent a secondary video stream reproduced insynchronization with a primary video stream represented by a main playitem, for example, when two pictures are combined.

In the picture-in-picture function, while video data contained in aprimary video stream (hereinafter referred to as primary video data) arebeing reproduced, video data contained in a secondary video stream(hereinafter referred to as secondary video data) can be simultaneouslyreproduced and superimposed thereon according to a user's operation.

Next, the case of which video data contained in video streams arereproduced from a disc having the foregoing data structure using thepicture-in-picture function will be described in brief. As shown in FIG.7, a picture of primary video data and a picture of secondary video dataare displayed, for example, on a display section disposed in an externaldisplay apparatus. In addition, an IG menu button 100 that switchesbetween START/STOP of the picture-in-picture function is displayed.

In this case, as shown in a flow chart of FIG. 8, when the useroperates, for example, a remote control commander and selects the IGmenu button 100 for the reproduction of secondary video data (at stepS11), a navigation command assigned to the selected IG menu button isinterpreted (at step S12). Thereafter, the navigation command assignedto the IG menu button, specifically a command that selects a secondaryvideo stream, is executed. As a result, the corresponding secondaryvideo data are selected and processed (at step S13). When the navigationcommand is executed, it is decided whether or not to select a secondaryvideo stream before the reproduction starts depending on whether or notthe secondary video stream is asynchronous with the primary videostream.

As shown in a flow chart of FIG. 9, the reproduction for the play listcorresponding to the secondary video data is started (at step S21).Thereafter, the IG menu button 100 to which the navigation command thatselects the secondary video data has been assigned is displayed (at stepS22). It is determined whether or not the secondary video data have beenselected based on the navigation command assigned to the IG menu button100 (at step S23). When the determined result denotes that the secondaryvideo data have been selected, the reproduction for the secondary videodata is started (at step S24). In contrast, when the determined resultdenotes that the secondary video data have not been selected, it isdetermined whether or not the secondary video data have been selected.

Next, a video processing method according to this embodiment of thepresent invention will be described in brief. It is assumed that whileprimary video data recorded by progressive scanning at a frame frequencyof 24 Hz (hereinafter referred to as 24p video data), 24p secondaryvideo data are reproduced at any timing using the picture-in-picturefunction. In this example, description of portions that do no directlyrelate to this embodiment of the present invention, such as audio data,will be omitted.

As shown in FIG. 10A, the 3-2 pull-down process is performed for 24pprimary video data to convert them into interlaced scanned primary videodata having a frame frequency of 30 Hz (field frequency of 60 Hz)(hereinafter referred to as 60i video data), A_(t), A_(b), A_(t), B_(b),B_(t), C_(b), C_(t), C_(b), and so forth. When secondary video data arecaused to be reproduced, the 3-2 pull-down process is performed for 24psecondary video data to convert them into 60i secondary video dataX_(b), X_(t), X_(b), Y_(t), Y_(b), Z_(t), Z_(b), Z_(t), and so forth.“t” and “b” of suffixes of data of each field represent a top field anda bottom field, respectively.

In the 3-2 pull-down process, three successive frames and two successiveframes of 24p video data are alternately output. Thus, video datagenerated in the 3-2 pull-down process have a period of five fields.Hereinafter, a set of n adjacent fields of n successive frames of 24pvideo data is referred to as a group. A group of three successive fieldsof three successive fields of 24p video data is referred to as a 3-fieldgroup, whereas a group of two adjacent fields of two successive framesof 24p video data is referred to as a 2-field group.

In FIG. 10A, frames of a 3-field group are hatched to identify them fromthose of a 2-field group. This notation representation applies tosimilar drawings.

When secondary video data are reproduced at any timing, as shown in FIG.10A, switching positions of groups of primary video data may bedifferent from those of secondary video data. Thus, for example, inregion a, while primary video data B_(b) and B_(t) are reproduced in theprimary video area, the top field and bottom field of secondary videodata reproduced in the secondary video area are Y_(t) and X_(b),respectively. In this case, since pictures of the top field and thebottom field are different from each other, so-called combing occurs inthe reproduced secondary video data as shown in FIG. 10B.

Three methods can be considered to prevent combing from occurring insecondary video data. In the first method, timing at which thereproduction of secondary video data is caused to start is moved totiming at which a picture of a top field of the next 3-field groupgenerated in the 3-2 pull-down process for the primary video data isoutput.

In the second method, the remaining number of pictures that have beendisplayed of a group generated by the 3-2 pull-down process for primaryvideo data that have been reproduced when the reproduction for secondaryvideo data is caused to start is set to the number of pictures that havebeen displayed of a group generated by the 3-2 pull-down process for thesecondary video data such that timing at which a picture of a top fieldof each group of the primary video data matches that of the secondaryvideo data.

In the third method, while pictures of a group generated by the 3-2pull-down process for primary video data are being displayed, picturesof a predetermined group of secondary video data are caused to bedisplayed and at timing of which groups of the primary video data arechanged, pictures that are displayed of the secondary video data arechanged such that timing at which the primary video data are changedmatch timing at which the secondary video data are changed.

According to this embodiment of the present invention, the first methodis used to prevent combing that occurs when secondary video data arereproduced. For example, as shown in FIG. 11, when timing at whichsecondary video data are caused to be reproduced does not match timingat which a picture of a top field of a 3-field group of primary videodata is output, if the secondary video data are reproduced, sincechanges of groups of the primary video data do not matched those of thesecondary video data, combing occurs when groups of the secondary videodata change.

In this case, in the first method, timing at which the reproduction ofsecondary video data is caused to start is moved by a predeterminingvalue and the secondary video data are reproduced at timing of which apicture of a top field of the next 3-field group of primary video datais output.

FIG. 12 shows an outline of a structure exemplifying a reproducingapparatus 1 according to this embodiment of the present invention. Thereare two types of reproducing apparatus according to this embodiment ofthe present invention, one of which is an “InMux” type reproducingapparatus that multiplexes a primary video stream and a secondary videostream to the same transport stream such that they synchronize with eachother and the other of which is an “OutMux” type reproducing apparatusthat multiplexes a primary video stream and a secondary video stream todifferent transport streams regardless of whether or not theysynchronize with each other. In this example, the “InMux” typereproducing apparatus, which multiplexes a primary video stream and asecondary video stream to the same transport stream such that theysynchronize with each other, will be described. In the followingdescription, the case of which frame frequencies of video data are to bechanged, for example video data having a frame frequency of 24 Hz areconverted into video data having a field frequency of 59.94 Hz, will bedescribed.

The reproducing apparatus 1 is composed of a disc drive 11, a buffer 12,a demultiplexer (DeMUX) 13, a primary video decoder 14, a primary videoconverting section 15, a secondary video decoder 16, a secondary videoconverting section 17, a controlling section 18, a system time clock(STC) circuit 19, and adding devices 20 and 21.

Stream data are recoded on a detachable record medium 10 that issupplied to the user. The record mediums 10 may include a read-only orrecordable disc-shaped record medium, a semiconductor memory such as aflash memory, and a detachable hard disk. The disc-shaped record mediumsmay include a Blu-ray Disc (registered trademark) and a Blu-ray Discread-only memory (BD-ROM) based on the Blu-ray Disc read-only standard,a digital versatile disc (DVD), and a compact disc (CD).

Stream data sources may include a communication network such as theInternet and a digital television broadcast besides the disc-shapedrecord mediums. In the following description, it is assumed that thestream data source is a detachable record medium 10 such as adisc-shaped record medium.

The disc drive 11 reproduces stream data from the loaded record medium10. Recorded on the detachable record medium 10 are stream data in whichvideo data and audio data have been multiplexed as a transport stream(TS) or a program stream (PS) defined, for example, in the MovingPicture Experts Group 2 Systems (MPEG2 System). A transport stream or aprogram stream reproduced from the record medium 10 is supplied to thebuffer 12. The buffer 12 supplies the stream data to the demultiplexer13 under the control of the controlling section 18 that will bedescribed later.

The demultiplexer 13 demultiplexes a primary video stream and asecondary video stream from the supplied stream data. If stream datahave been supplied as a transport stream from the buffer 12, thedemultiplexer 13 analyzes the packet identification (PID) of eachtransport packet and collects packets for primary video data. Packetizedelementary stream (PES) packets are reconstructed from data stored inpayloads of the collected packets. An elementary stream of primary videodata is obtained from each PES packet and thereby a primary video streamis restored. Like primary video stream, the demultiplexer 13 collectspackets of secondary video data from transport packets based on theirPIDs and thereby restores a secondary video stream.

If stream data of a program stream have been supplied from the buffer12, the demultiplexer 13 separates PES packets from the stream databased on header information of pack headers or the like, extracts anelementary stream of video data from PES packets based on informationstored in headers of the PES packets, and restores a video stream. Likevideo stream, the demultiplexer 13 separates PES packets from the streamdata based on header information such as pack headers and restores anaudio stream.

The primary video stream separated by the demultiplexer 13 is suppliedto the primary video decoder 14 and the secondary video stream to thesecondary video decoder 16. The primary video decoder 14 decodes thesupplied primary video stream and outputs base-band video data.

As mentioned above, the encoding system for a video stream transmittedis not limited to the MPEG2 System. Encoding systems for a video streaminclude those specified in International TelecommunicationUnion-Telecommunication Standardization Sector (ITU-T) RecommendationH.264, International Organization for Standardization(ISO)/International Electrotechnical Commission (IEC) InternationalStandard 14496-10 (MPEG-4 part 10) Advanced Video Coding (hereinafterreferred to as MPEG4 AVC), and Video Codec 1 (VC-1) System. A videostream encoded in such a system can be transmitted according to theMPEG2 System.

The primary video decoder 14 can decode a primary video stream accordingto each of these encoding formats. In this case, the primary videodecoder 14 determines the encoding format based on, for example, headerinformation of the primary video stream and attribute information addedto the primary video stream in a predetermined manner and decodes theprimary video stream according to the determined decoding system. Anencoding format may be set by an external control based on user'soperation or the like.

Of course, the primary video decoder 14 may deal with only a singleencoding format. In this case, if the primary video decoder 14 issupplied with a video stream having been encoded in an encoding formatit is unable to deal therewith, for example an error process isperformed.

In addition, the primary video decoder 14 obtains the frame rate, namelythe frame frequency, of the supplied primary video stream. For example,according to the foregoing MPEG2 System, MPEG4 AVC System, and VC-1System, information regarding the frame rate is stored in the headerportion of the stream. The primary video decoder 14 extracts informationregarding the frame rate, for example, from the header portion of thesupplied primary video stream and obtains the frame rate of the videostream therefrom.

More specifically, according to the MPEG2 System, frame rateframe_rate_value is described in data item frame_rate_code of a sequenceheader. According to the MPEG4 AVC System, frame rate frame_rate_valueis obtained by calculating (time_scale/num_units_in_tick)/2 usingparameter num_units_in_tick and parameter time_scale of sequenceparameter vui_parameters( ) of a sequence parameter set. According tothe VC-1 System, frame rate frame_rate_value is obtained by calculating(value of Frame Rate Numerator)/(value of Frame Rate Denominator) usingvalue “value of Frame Rate Numerator” and value “value of Frame RateDenominator” represented by item FRAMERATENR and item FRAMERATEDR ofsequence layer SEQUENCELAYER( ).

Primary video data into which a primary video stream has been decodedand frame rate information that has been obtained from the primary videostream in a predetermined manner are supplied from the primary videodecoder 14 to the primary video converting section 15. The primary videodata are supplied to the primary video converting section 15, forexample, at the frame rate of the video data.

Like the primary video decoder 14, the secondary video decoder 16 candecode a primary video stream according to each of the plurality ofencoding formats. In addition, the secondary video decoder 16 extractsinformation regarding the frame rate stored in the header portion of thesupplied secondary video stream and obtains the frame rate of the videostream.

Secondary video data into which the secondary video stream has beendecoded and the frame rate information that has been obtained from thesecondary video stream in a predetermined manner are supplied from thesecondary video decoder 16 to the secondary video converting section 17.The secondary video data are supplied to the secondary video convertingsection 17, for example, at the frame rate of the video data.

The STC circuit 19 generates an STC that is a synchronous referencesignal that causes primary video data and secondary video data tosynchronize with each other. The generated STC is supplied to thesecondary video converting section 17 through the primary videoconverting section 15 and the adding device 20.

The controlling section 18 is composed of a central processing unit(CPU), a read-only memory (ROM), a random access memory (RAM), and soforth. The ROM pre-stores a program that is executed on the CPU and datanecessary for operation. The RAM is used as a work memory of the CPU.The CPU reads the program and data from the ROM when necessary andcontrols each section of the reproducing apparatus 1 using the RAM as awork memory. The program and data stored in the ROM can be rewritten andupdated.

The controlling section 18 controls the STC circuit 19 to delay the STCvalue supplied to the secondary video converting section 17 by apredetermined value through the adding device 20 such that the secondaryvideo data that are output are delayed by the predetermined value.

The primary video converting section 15 converts the frame frequency ofprimary video data based on the frame rate information supplied from theprimary video decoder 14. For example, as shown in FIG. 13A, the primaryvideo converting section 15 is composed of a frame memory 22 and aprimary video controlling section 23. The primary video controllingsection 23 controls reading of the primary video data from the framememory 22 under the control of the controlling section 18 to performthose including the 3-2 pull-down process.

In the 3-2 pull-down process, which converts, for example, 24p primaryvideo data into 60i primary video data, 24p primary vide data, namelyprimary video data supplied at a frame frequency of 24 Hz, are stored inthe frame memory 22 for each frame and the primary video controllingsection 23 controls reading of successive three frames or successive twoframes from the frame memory 22.

The converted primary video data are output only when the STC value asthe synchronous reference signal supplied from the STC circuit 19matches a time represented by a presentation time stamp (PTS) of theprimary video data. The PTS is a time stamp that represents areproduction time of a picture or a sound as 33 bits of a 90 kHz clockvalue. The PTS is added to each reproduction unit that is called anaccess unit. The PTS is added to the header portion of a packet thatstores a top portion of the access unit.

Like the primary video converting section 15, the secondary videoconverting section 17 converts the frame frequency of the secondaryvideo data based on the frame rate information supplied from thesecondary video decoder 16. The secondary video converting section 17converts the size of the secondary video data.

For example, as shown in FIG. 13B, the secondary video convertingsection 17 is composed of a frame memory 24, a scaling section 25, and asecondary video controlling section 26. The secondary video controllingsection 26 controls reading of secondary video data from the framememory 24 under the control of the controlling section 18 to performthose including the 3-2 pull-down process.

The secondary video data that are output from the frame memory 24 aresupplied to the scaling section 25. The scaling section 25 converts thesize of the secondary video data to a predetermined size under thecontrol of the secondary video controlling section 26 and outputs theresultant secondary video data. Specifically, for example, the scalingsection 25 converts the size of the secondary video data such that thesize of the secondary video data becomes 1 fold, 1.5 fold, ½ fold, or ¼fold the original size. The scaling section 25 may be able to convertthe size of the supplied secondary video data to the size of the fullscreen.

Returning to FIG. 12, the adding device 21 combines the primary videodata that are output from the primary video converting section 15 andthe secondary video data that are output from the secondary videoconverting section 17 for each frame and outputs the combined videodata.

It was assumed that each section of the reproducing apparatus 1 shown inFIG. 12 is composed of hardware. However, this embodiment of the presentinvention is not limited to such an example. In other words, all or partof the demultiplexer 13, the primary video decoder 14, the primary videoconverting section 15, the secondary video decoder 16, the secondaryvideo converting section 17, the controlling section 18, and the STCcircuit 19 of the reproducing apparatus 1 may be structured by executinga predetermined program on the CPU. Instead, the program may bepre-stored in a ROM (not shown) of the reproducing apparatus 1. Instead,the program may be provided as a record medium such as a DVD-ROM or aCD-ROM. Instead, the program may be provided through a communicationnetwork such as the Internet. The provided program is stored in a harddisk drive (not shown) of the reproducing apparatus 1 and then read andexecuted by the CPU.

Next, combinations of primary video data and secondary video datacombined when the picture-in-picture function is used will be described.Encoding formats for primary video data and secondary video data includethe MPEG2 System, MPEG4 AVC System, and VC-1 System as described above.When primary video data and secondary video data are combined using thepicture-in-picture function, combinations of primary video data andsecondary video data have been set.

For example, as shown in FIG. 14, primary video data encoded accordingto the MPEG2 System can be combined with secondary video data encodedaccording to any of the MPEG2 System, the MPEG4 AVC System, and the VC-1System. On the other hand, primary video data encoded according to theMPEG4 AVC System can be combined with secondary video data encodedaccording to only the MPEG4 AVC System. On the other hand, primary videodata encoded according to the VC-1 System can be combined with secondaryvideo data encoded according to only the VC-1 System.

Likewise, combinations of sizes, frame rates, and scanning systems ofprimary video data and secondary video data that can be combined havebeen set. For example, as shown in FIG. 15, primary video data whose thepicture size (horizontal size×vertical size) is 1920×1080 [pixels] andthat are interlaced scanned at a frame rate of 29.97 [Hz] can becombined with secondary video data whose picture size is 720×480[pixels], 1920×1080 [pixels], or 1440×1080 [pixels] and that areinterlaced scanned at a frame rate of 29.97 [Hz].

Next, a reproducing method according to this embodiment of the presentinvention will be described.

According to this embodiment of the present invention, when secondaryvideo data are reproduced using the picture-in-picture function, controlis performed such that timing at which the reproduction of the secondaryvideo data starts matches timing at which a picture of a top field of a3-field group of primary video data is displayed.

Next, with reference to a flow chart of FIG. 16, a process thatcalculates timing at which the reproduction of secondary video datastarts will be described. The process exemplified in FIG. 16 is mainlyperformed by the controlling section 18 of the reproducing apparatus 1.

At step S31, the user operates the remote control commander or the liketo start reproducing secondary video data. At step S32, the STC value attiming of which the reproduction of the secondary video data starts isobtained and stored in value cur_stc that represents the current STCvalue.

Thereafter, at step S33, value in_pts that represents an IN point of amain play item corresponding to the primary video data represented bythe current STC value cur_stc is obtained and stored in valueprimary_video_datum_pts. At step S34, value in_pts, which represents anIN point of a sub play item corresponding to the secondary video data,is obtained and stored in value async_pinp_in_pts.

At step S35, a predetermined value, for example value “300×90”corresponding to around 300 msec is stored in value start_margin thatrepresents a margin for which the reproduction of the secondary videodata starts a predetermined time after the current time. This valuerepresents a value of which 300 msec is converted regarding 90 kHz ofthe STC clock.

At step S36, PTS value tmp_start_pts that represents a temporaryreproduction start position of secondary video data is calculated basedon margin start_margin for which the reproduction of the secondary videodata starts according to formula (1).

tmp_start_pts=cur_stc+start_margin  (1)

Thereafter, at step S37, difference diff of the PTS values thatrepresent the reproduction start positions of the primary video data andthe secondary video data is calculated based on PTS valueprimary_video_datum_pts that represents the reproduction start positionof the primary video data and PTS value tmp_start_pts that representsthe temporary reproduction start position of the secondary video dataaccording to formula (2).

diff=tmp_start_pts−primary_video_datum_pts  (2)

At step S38, the frame rate of the primary video data is obtained andstored in value frame_duration. At step S39, the length twice frame rateframe_duration of the primary video data contained in difference diff ofthe PTS values, which represent the reproduction start positions of theprimary video data and secondary video data, calculated at step S37, iscalculated according to formula (3) and the integer part of thecalculated result is stored in value num.

num=diff/(frame_duration×2)  (3)

At step S40, difference diff_grid_x2 of which difference diff of the PTSvalues, which represent the reproduction start positions of the primaryvideo data and the secondary video data, is normalized with a valuetwice frame rate frame_duration of the primary video data is calculatedaccording to formula (4). This difference diff_grid_x2 becomes a delayvalue of the STC value supplied from the controlling section 18 to theadding device 20 against the STC value supplied from the STC circuit 19.

diff_grid_x2=num×(frame_duration×2)  (4)

At step S41, PTS value async_pinp_start_pts that represents thereproduction start time of the secondary video data is calculatedaccording to formula (5). In other words, the secondary video data arecaused to be reproduced from the start position of combinations of3-field groups and 2-field groups generated when the 3-2 pull-downprocess is performed for the primary video data.

async_pinp_start_pts=primary_video_datum_pts+diff_grid_x2  (5)

Next, with reference to a flow chart shown in FIG. 17, the reproducingprocess according to this embodiment of the present invention will bedescribed. At step S51, it is determined whether or not video data to beprocessed are secondary video data. When the determined result denotesthat the video data to be treated are secondary video data, the flowadvances to step S52.

At step S52, in_pts of a sub play item corresponding to the secondaryvideo data is obtained. At step S53, in_pts obtained at step S52 issubtracted from the PTS of the secondary video data and thenasync_pinp_start_pts is added to the subtracted result to correct thePTS of the secondary video data. Thereafter, the flow advances to stepS54.

In contrast, when the determined result at step S51 denotes that thevideo data to be processed are not secondary video data, namely primaryvideo data, the flow advances to step S54.

At step S54, a time represented by the PTS of the video data and the STCvalue are compared to set timing at which the video data are displayed.When the video data to be processed are secondary video data, the PTSvalue that has been corrected at step S53 is used. When the timerepresented by the PTS of the video data matches the STC value at stepS55, a picture is displayed.

Thereafter, at step S56, the number of Vsync's is counted based on valueprsn_vsync_cnt that represents the number of Vsync's for pictures thathave been displayed and it is determined whether or not pictures havebeen displayed for the number of Vsync's. When the determined resultdenotes that pictures have been displayed for value prsn_vsync_cnt,which represents the number of Vsync's, the flow advances to step S57.At step S57, the next picture is displayed. The method of calculatingprsn_vsync_cnt, which represents the number of Vsync's, will bedescribed later.

In contrast, when the determined result denotes that pictures have notbeen displayed for prsn_vsync_cnt, which represents the number ofVsync's, the flow returns to step S56. At step S56, it is determinedwhether or not pictures have been displayed for prsn_vsync_cnt, whichrepresents the number of Vsync's.

Next, the method of calculating value prsn_vsync_cnt, which representsthe number of Vsync's for pictures that have been displayed, used at theforegoing step S56 will be described. With reference to a flow chartshown in FIG. 18, the process that calculates native_vsync count, whichrepresents the original number of Vsync's for pictures that have beendisplayed, will be described. At step S61, the difference between thePTS value of a picture that is being displayed and the PST value of amain play item is stored in value pts_diff.

Thereafter, at step S62, it is determined whether or not the frame rateof the picture that is being displayed is 23.976 Hz. When the determinedresult denotes that the frame rate of the picture that is beingdisplayed is 23.976 Hz, the flow advances to step S63. At step S63, avalue that represents the Vsync interval of 59.94 Hz is stored inframe_rate. The value stored in frame_rate is for example a valueconverted regarding 90 kHz of the STC clock. Specifically, “90 kHz/59.94Hz=1501.5” is stored in frame_rate.

At step S64, value x is calculated according to formula (6) and theinteger part of the calculated result is stored in value x. At step S65,value native_vsync_count that represents the original number of Vsync'sfor pictures that have been displayed is calculated according to formula(7) and the integer part of the calculated result is stored in valuenative_vsync_count.

x=(pts_diff×2)/(frame_rate×5)  (6)

native_vsync_count=x×5/2  (7)

At step S66, it is determined whether or not value x calculated at stepS64 is odd. When the determined result denotes that value x is odd, theflow advances to step S67. At step S67, “1” is added to the value ofnative_vsync_count calculated at step S65 and then the process iscomplete. When the determined result denotes that value x is not odd,namely even, the process is complete.

In contrast, when the determined result at step S62 denotes that theframe rate of the picture that is being displayed is not 23.976 Hz, theflow advances to step S68. At step S68, it is determined whether or notthe frame rate of the picture that is being displayed is 24 Hz. When thedetermined result denotes that the frame rate of the picture that isbeing displayed is 24 Hz, the flow advances to step S69. At step S69, avalue that represents the Vsync interval of 60 Hz is stored inframe_rate. The value that is stored in value frame_rate is a valueconverted regarding 90 kHz of the clock of the STC. Specifically “90kHz/60 Hz=1500” is stored in value frame_rate.

In contrast, when the determined result at step S69 denotes that theframe rate of the picture that is being displayed is not 24 Hz, the flowadvances to step S70. At step S70, it is determined whether or not theframe rate of the picture that is being displayed is 29.97 Hz or 59.94Hz. When the determined result denotes that the frame rate of thepicture that is being displayed is 29.97 Hz or 59.94 Hz, the flowadvances to step S71.

At step S71, a value that represents the Vsync interval of 59.94 Hz isstored in frame_rate. The value stored in value frame_rate is a valueconverted regarding 90 kHz of the clock of the STC. Specifically, “190kHz/59.94 Hz=1501.5” is stored in value frame_rate.

In contrast, when the determined result at step S70 denotes that theframe rate of the picture that is being displayed is neither 29.97 Hznor 59.94 Hz, the flow advances to step S72. At step S72, a value thatrepresents the Vsync interval of 60 Hz is stored. The value stored inframe_rate is a value converted regarding 90 kHz of the clock of theSTC. Specifically, “90 kHz/60 Hz=1500” is stored in frame_rate.

At step S73, value native_vsync_count that represents the originalnumber of Vsync's for pictures that have been displayed is calculatedaccording to formula (8).

native_vsync_count=pts_diff/frame_rate  (8)

Next, with reference to a flow chart shown in FIG. 19, the process thatcalculates value prsn_vsync_cnt that represents the number of Vsync'sfor pictures that have been displayed will be described. At step S81, itis determined whether or not the frame rate of the picture that is beingdisplayed is 23.976 Hz. When the determined result denotes that theframe rate of the picture that is being displayed is 23.976 Hz, the flowadvances to step S82.

At step S82, it is determined whether or not a value of which valuenative_vsync_count, which represents the original number of Vsync's forpictures that have been displayed, is multiplied by ⅖ is odd.

When the determined result represents that the result is odd, the flowadvances to step S83. At step S83, “2” is set to value prsn_vsync_cntthat represents the number of Vsync's for pictures that have beendisplayed, “1501.5/2” is added to the PTS of the picture that is beingdisplayed, and “2” is set to value vsync_cnt.

In contrast, when the determined result denotes that the value of whichvalue native_vsync_count, which represents the original number ofVsync's for pictures that have been displayed, is multiplied by ⅖ is notodd, the flow advances to step S84. At step S84, “3” is set to valueprsn_vsync_cnt, which represents the number of Vsync's for pictures thathave been displayed, and “3” is set to value vsync_cnt.

When the determined result at step S81 denotes that the frame rate ofthe picture that is being displayed is not 23.976 Hz, the flow advancesto step S85. At step S85, it is determined whether or not the frame rateof the picture that is being displayed is 24 Hz. When the determinedresult denotes that the frame rate of the picture that is beingdisplayed is 24 Hz, the flow advances to step S86.

At step S86, it is determined whether or not a value of which valuenative_vsync_count, which represents the original number of Vsync's forpictures that have been displayed, is multiplied by ⅖ is odd.

When the determined result denotes that the result is odd, the flowadvances to step S87. At step S87, “2” is set to value prsn_vsync_cntthat represents the number of Vsync's for pictures that have beendisplayed, “1501.5/2” is added to the PTS of the picture that is beingdisplayed, and “2” is set to value vsync_cnt.

In contrast, when the determined result at step S86 denotes that theresult of which value native_vsync_count, which represents the originalnumber of Vsync's for pictures that have been displayed, is multipliedby ⅖ is not odd, the flow advances to step S88. At step S88, “3” is setto value vsync_cnt.

At step S89, value vsync_cnt decided at step S87 or S88 is stored invalue prsn_vsync_cnt that represents the number of Vsync's for picturesthat have been displayed and then the process is complete.

In contrast, when the determined result at step S85 denotes that theframe rate of the picture that is being displayed is not 24 Hz, the flowadvances to step S90. At step S90, the original number of Vsync's forpictures that have been displayed is stored in value vsync_cnt accordingto the specifications of the video decoder. At step S91, value vsync_cntdecided at step S90 is stored in value prsn_vsync_cnt, which representsthe number of Vsync's for pictures that have been displayed.

At step S92, it is determined whether or not the frame rate of thepicture that is being displayed is 29.97 Hz or 59.94 Hz. When thedetermined result denotes that the frame rate of the picture that isbeing displayed is 29.97 Hz or 59.94 Hz, the process is complete.Likewise, when the determined result denotes that the frame rate of thepicture that is being displayed is neither 29.97 Hz nor 59.94 Hz, theprocess is complete.

Thus, according to this embodiment of the present invention, timing atwhich the reproduction of secondary video data starts is calculated. Bymatching this timing with timing at which a top field of a 3-field groupgenerated by the 3-2 pull-down process for the primary video data isoutput next time, combing that occurs when the secondary video data arereproduced can be prevented.

Next, a first modification of this embodiment of the present inventionwill be described. According to the first modification of thisembodiment of the present invention, the foregoing second method is usedto prevent combing that occurs when secondary video data are reproduced.

In the second method, the remaining number of pictures that have beendisplayed of a group generated by the 3-2 pull-down process for primaryvideo data that have been reproduced when the reproduction for secondaryvideo data is caused to start is set to the number of pictures that havebeen displayed of a group generated by the 3-2 pull-down process for thesecondary video data such that timing at which a picture of a top fieldof each group of the primary video data matches that of the secondaryvideo data.

For example, as shown in FIG. 20, since the remaining number of picturesof a group of primary video data when the reproduction of secondaryvideo data is caused to start is “2”, “2” is set to the number ofpictures displayed of the first group of the secondary video data. Thus,timing at which groups of the primary video data are changed matchestiming at which groups of the secondary video data are changed.

Next, the reproducing method according to the first modification of thisembodiment of the present invention will be described. With reference toa flow chart shown in FIG. 21, the process that calculates timing atwhich the reproduction of secondary video data starts will be described.The process exemplified in FIG. 21 is mainly performed by thecontrolling section 18 of the reproducing apparatus 1.

At step S101, when the user operates the remote control commander or thelike, secondary video data are caused to be reproduced. At step S102,the STC value at which the reproduction of secondary video data startsis obtained and stored in value cur_stc that represents the current STCvalue.

At step S103, a predetermined value, for example, value “300×90” thatcorresponds to around 300 msec is stored in value start_margin thatrepresents a margin for which the reproduction of secondary video datastarts a predetermined time after the current time. This valuerepresents a value of which 300 msec is converted regarding 90 kHz ofthe STC clock. This margin start_margin becomes a delay of the STC valuesupplied from the controlling section 18 through the adding device 20against the STC value supplied from the STC circuit 19.

At step S104, PTS value async_pinp_start_pts that represents areproduction start time of secondary video data is calculated based onmargin start_margin for which the reproduction of secondary video datastarts is calculated according to formula (9).

async_pinp_start_pts=cur_stc+start_margin  (9)

At step S105, PTS value async_pinp_start_pts, which represents thereproduction start time of the secondary video data, is set as thereproduction start time of the secondary video data to the STC circuit19.

Next, with reference to a flow chart shown in FIG. 22, the reproducingprocess according to the first modification of this embodiment will bedescribed. At step S111, it is determined whether or not video data tobe processed are secondary video data. When the determined resultdenotes that the video data to be processed are secondary video data,the flow advances to step S112.

At step S112, in_pts of a sub play item corresponding to the secondaryvideo data is obtained. At step S113, in_pts obtained at step S112 issubtracted from the PTS of the secondary video data andasync_pinp_start_pts is added to the subtracted result so as to correctthe PTS of the secondary video data. Thereafter, the flow advances tostep S114.

When the determined result at step S111 denotes that the video data tobe processed are not secondary video data, namely primary video data,the flow advances to step S114.

At step S114, a time represented by the PTS of the video data and theSTC value are compared to set timing at which the video data aredisplayed. When the video data to be processed are secondary video data,the PTS value corrected at step S113 is used.

At step S115, it is determined whether or not the video data to beprocessed are secondary video data. When the determined result denotesthat the video data to be processed are secondary video data, the flowadvances to step S116.

At step S116, value prsn_vsync_cnt that represents the number of Vsync'sfor pictures that have been displayed of the primary video data is setto value prsn_vsync_cnt that represents the number of Vsync's forpictures that have been displayed of the secondary video data. Valueprsn_vsync_cnt, which represents the number of Vsync's for pictures thathave been displayed, can be calculated based on the flow charts shown inFIG. 18 and FIG. 19.

In contrast, when the determined result at step S115 denotes that thevideo data to be processed are not the secondary video data, the flowadvances to step S117. At step S117, when a time represented by the PTSof the video data matches the STC value, a picture is displayed.

Thereafter, at step S118, the number of Vsync's is counted based onvalue prsn_vsync_cnt, which represents the number of Vsync's forpictures that have been displayed and it is determined whether or notpictures have been displayed for the number of Vsync's. When thedetermined result denotes that pictures have been displayed forprsn_vsync_cnt, which represents the number of Vsync's, the flowadvances to step S119. At step S119, the next picture is displayed.

In contrast, when the determined result denotes that pictures have notbeen displayed for prsn_vsync_cnt, which represents the number ofVsync's, the flow returns to step S118. At step S118, it is determinedwhether or not pictures have been displayed for prsn_vsync_cnt, whichrepresents the number of Vsync's.

According to the first modification of this embodiment of the presentinvention, the remaining number of picture that have been displayed of agroup generated by the 3-2 pull-down process for the primary video datathat have been reproduced when the reproduction of the secondary videodata is caused to start is obtained and set to the number of picturesthat have been displayed of a group generated by the 3-2 pull-downprocess for the secondary video data. Thus, timing at which a picture ofa top field of each group of the primary video data can be matched witha picture of a top field of each group of the secondary video data. As aresult, coming that occurs when the secondary video data are reproducedcan be prevented.

Next, a second modification of this embodiment of the present inventionwill be described. In the second modification of this embodiment of thepresent invention, the foregoing third method is used to preventcombining that occurs when secondary video data are reproduced.

In the third method, while pictures of a group generated by the 3-2pull-down process for primary video data are being displayed, picturesof a predetermined group of secondary video data are caused to bedisplayed and at timing of which groups of the primary video data arechanged, pictures that are displayed of the secondary video data arechanged such that timing at which the primary video data are changedmatch timing at which the secondary video data are changed.

For example, as shown in FIG. 23, picture X of secondary video datadisplayed when group A is changed to group B of groups generated by the3-2 pull-down process for the primary video data is kept displayed whilepictures of group B of primary video data are being displayed. Likewise,picture Y of secondary video data displayed when group B is changed togroup C of groups of primary video data is kept displayed while picturesof group C of the primary video data are being displayed.

In other words, while a picture of a predetermined group of the primaryvideo data is being reproduced, a picture of the secondary video datadisplayed when a picture of a top field of the predetermined group isdisplayed is kept displayed. When groups of the primary video data arechanged, a picture of a top field of the next group of the secondaryvideo data is displayed.

Next, the reproducing method according to the second modification ofthis embodiment of the present invention will be described. Withreference to a flow chart shown in FIG. 24, a process that calculatestiming at which the reproduction of the secondary video data starts willbe described. The process exemplified in FIG. 24 is mainly performed bythe controlling section 18 of the reproducing apparatus 1.

At step S121, the user operates the remote control commander or the liketo start reproducing the secondary video data. At step S122, the STCvalue at which the reproduction of the secondary video data started andstored in cur_stc that represents the current STC value.

At step S123, a predetermined value, for example value “300×90”corresponding to for example 300 msec is stored in value start_marginthat represents a margin for which the reproduction of the secondaryvideo data starts a predetermined time after the current time. Thisvalue represents a value of which 300 msec is converted regarding 90 kHzof the STC clock. This margin start_margin becomes a delay of the STCvalue supplied from the controlling section 18 through the adding device20 against the STC value supplied from the STC circuit 19.

At step S124, PTS value async_pinp_start_pts that represents areproduction start time of the secondary video data is calculated basedon margin start_margin for which the reproduction of the secondary videodata starts according to formula (10).

async_pinp_start_pts=cur_stc+start_margin  (10)

At step S125, PTS value async_pinp_start_pts, which represents thereproduction start time of the secondary video data, is set as areproduction start time of the secondary video data to the STC circuit19.

Next, with reference to a flow chart shown in FIG. 25, the reproducingprocess according to the second modification of this embodiment will bedescribed. At step S131, it is determined whether or not video data tobe processed are secondary video data. When the determined resultdenotes that the video data to be processed are secondary video data,the flow advances to step S132.

At step S132, in_pts of a sub play item corresponding to the secondaryvideo data is obtained. At step S133, in_pts obtained at step S132 issubtracted from the PTS of the secondary video data andasync_pinp_start_pts is added to the subtracted result so as to correctthe PTS of the secondary video data. Thereafter, the flow advances tostep S134.

In contrast, when the determined result at step S131 denotes that thevideo data to be processed are not secondary video data, namely primaryvideo data, the flow advances to step S134.

At step S134, the time represented by the PTS of the video data iscompared with the STC value so as to set timing at which the video dataare displayed. When the video data to be processed are secondary videodata, the PTS value corrected at sep S133 is used. At step S135, apicture of the video data is output to the display frame memory.

At step S136, it is determined whether or not the video data to beprocessed are primary video data. When the determined result denotesthat the video data to be processed are primary video data, the flowadvances to step S137. At step S137, a picture is obtained from thedisplay frame memory for the secondary video data and overwritten to thedisplay frame memory for the primary video data to combine pictures thatare displayed.

In contrast, when the determined result as step S136 denotes that thevideo data to be processed are not primary video data, the flow advancesto step S138. At step S138, the number of Vsync's is counted based onvalue prsn_vsync_cnt that represents the number of Vsync's for picturethat have been displayed and it is determined whether or not pictureshave been displayed for the number of Vsync's. When the determinedresult denotes that pictures have been displayed for prsn_vsync_cnt,which represents the number of Vsync's, the flow advances to step S139.At step S139, the next picture is displayed.

In contrast, when the determined result denotes that pictures have notbeen displayed for prsn_vsync_cnt, which represents the number ofVsync's, the flow returns to step S138. At step S138, it is determinedwhether or not pictures have been displayed for prsn_vsync_cnt, whichrepresents the number of Vsync's. Value prsn_vsync_cnt, which representsthe number of Vsync's, can be calculated based on the flow charts shownin FIG. 18 and FIG. 19.

According to the second modification of this embodiment of the presentinvention, while pictures of a predetermined group of the primary videodata are being displayed, a picture of secondary video data displayedwhen a picture of a top field of the group is displayed is keptreproduced and when groups of the primary video data are changed, apicture that is displayed of the secondary video data is changed to apicture of a top field of the next group. Thus, timing at which apicture of a top field of each group of the primary video data can bematched with timing at which a picture of a top field of each group ofthe secondary video data. As a result, combing that occurs when thesecondary video data are reproduced can be prevented.

One embodiment of the present invention and a first modification andsecond modification thereof have been described. Instead, it should beunderstood by those skilled in the art that various modifications,combinations, sub-combinations and alternations may occur depending ondesign requirements and other factors insofar as they are within thescope of the appended claims or the equivalents thereof. For example,the case of which a primary video stream and a second video stream aresupplied from the same record medium to the reproducing apparatus 1 hasbeen described. Instead, the embodiment of the present invention and thefirst modification and second modification thereof can be applied to thecase of which a primary video stream and a secondary video stream aresupplied from different record mediums.

FIG. 26 shows a structure exemplifying a reproducing apparatus 1′ towhich first stream data containing a primary video stream and secondstream data containing a secondary video stream are supplied fromdifferent record mediums. The reproducing apparatus 1′ is composed of adisc drive 31, a buffer 32, a demultiplexer (DeMUX) 33, a record medium41, a buffer 42, a demultiplexer (DeMUX) 43, a primary video decoder 14,a primary video converting section 15, a secondary video decoder 16, asecondary video converting section 17, a controlling section 18, an STCcircuit 19, and adding devices 20 and 21. In the reproducing apparatus1′ shown in FIG. 26, sections similar to those of the reproducingapparatus 1 will be denoted by similar reference numerals and theirdescription will be omitted.

The first stream data are recorded to a detachable record medium 30 thatis supplied to the user. The record mediums 30 may include a disc-shapedrecord medium such as a BD-ROM.

The disc drive 31 reproduces first stream data from the record medium 30loaded into the disc drive 31. First stream data have been recorded onthe record medium 30 in the form of a transport stream (TS) or a programstream (PS), for example, defined in the MPEG2 System. In the firststream data, video data and audio data have been multiplexed. Thetransport stream or the program stream reproduced from the record medium30 is supplied to the buffer 32. The buffer 32 supplies the first streamdata to the demultiplexer 33 under the control of the controllingsection 18.

The demultiplexer 33 separates a primary video stream from the suppliedfirst stream data. When first stream data are supplied as a transportstream from the demultiplexer 33, the demultiplexer 33 analyzes the PIDfor each transport packet, collects packets for primary video data, andrestores the primary video stream.

On the other hand, second stream data have been downloaded through acommunication network such as the Internet and then recorded on therecord medium 41. The record mediums 41 may include a semiconductormemory such as a flash memory and a detachable hard disk drive (HDD).

A transport stream or a program stream that is output from the a recordmedium 41 is supplied to the buffer 42. The buffer 42 supplies thesecond stream data to the demultiplexer 43 under the control of thecontrolling section 18.

The demultiplexer 43 separates a secondary video stream from the secondstream data. When second stream data are supplied as a transport streamfrom the buffer 42, the demultiplexer 13 analyzes the PID of eachtransport packet, collects packets for the secondary video data, andrestores the secondary video stream.

1. A reproducing apparatus which simultaneously reproduces a first videosignal and a second video signal reproduced at any timing against thefirst video signal, comprising: a first converting section which outputsthe first video signal having a first frame frequency alternately forthree successive frames and two successive frames so as to convert thefirst video signal having the first frame frequency into the first videosignal having a second frame frequency, the first frame frequency andthe second frame frequency having a relationship of 2 to 5; a secondconverting section which outputs the second video signal having thefirst frame frequency alternately for three successive frames and twosuccessive frames so as to convert the second video signal having thefirst frame frequency into the second video signal having the secondframe frequency, the first frame frequency and the second framefrequency having a relationship of 2 to 5; and a controlling sectionwhich performs control such that timing at which a field group based onfields of the three successive frames of the first video signal havingthe second frame signal is changed to a field group based on fields ofthe two successive frames thereof matches timing at which a field groupbased on fields of the three successive frames of the second videosignal having the second frame signal is changed to a field group basedon fields of the two successive frames thereof.
 2. The reproducingapparatus as set forth in claim 1, wherein the controlling sectioncalculates timing at which the reproduction of a top field of the fieldgroup of fields of the three successive frames of the first video signalhaving the second frame frequency starts when the second video signal isreproduced at any timing, and wherein the controlling section matchesthe calculated timing and timing at which the reproduction of a topfield of the field group of the fields of the three successive frames ofthe second video signal having the second frame frequency.
 3. Thereproducing apparatus as set forth in claim 1, wherein the controllingsection calculates remaining numbers of fields of the field groups basedon the fields of the three successive frames and the two successiveframes of the first video signal having the second frame frequency whenthe second video signal is reproduced at any timing, and wherein thecontrolling section matches the number of fields of the field groupbased on the fields of the first three successive frames or the firsttwo successive frames of the second video signal having the second framefrequency with the remaining numbers of fields.
 4. The reproducingapparatus as set forth in claim 1, wherein the controlling section keepsreproducing the fields of the three successive frames and two successiveframes of the second video signal having the second frame frequency attiming of which a top field of the field group of the fields of thethree successive frames and the two successive frames of the first videosignal having the second frequency is reproduced when the second videosignal is reproduced at any timing and the field groups based on thefields of the three successive frames and two successive frames of thefirst video signal having the second frame frequency are changed.
 5. Thereproducing apparatus as set forth in claim 1, wherein the first videosignal and the second video signal have been recorded on the same recordmedium.
 6. The reproducing apparatus as set forth in claim 5, whereinthe first video signal and the second video signal have been recorded ona BD-ROM.
 7. The reproducing apparatus as set forth in claim 1, whereinthe first video signal and the second video signal have been recorded ondifferent record mediums.
 8. The reproducing apparatus as set forth inclaim 7, wherein the first video signal is recorded on a BD-ROM.
 9. Areproducing method of simultaneously reproducing a first video signaland a second video signal reproduced at any timing against the firstvideo signal, comprising the steps of: outputting the first video signalhaving a first frame frequency alternately for three successive framesand two successive frames so as to convert the first video signal havingthe first frame frequency into the first video signal having a secondframe frequency, the first frame frequency and the second framefrequency having a relationship of 2 to 5; outputting the second videosignal having the first frame frequency alternately for three successiveframes and two successive frames so as to convert the second videosignal having the first frame frequency into the second video signalhaving the second frame, the first frame frequency and the second framefrequency having a relationship of 2 to 5; and performing control suchthat timing at which a field group based on fields of the threesuccessive frames of the first video signal having the second framesignal is changed to a field group based on fields of the two successiveframes thereof matches timing at which a field group based on fields ofthe three successive frames of the second video signal having the secondframe signal is changed to a field group based on fields of the twosuccessive frames thereof.
 10. A recording medium storing a reproducingprogram which causes a computer apparatus to execute a reproducingmethod of simultaneously reproducing a first video signal and a secondvideo signal reproduced at any timing against the first video signal,the reproducing method comprising the steps of: outputting the firstvideo signal having a first frame frequency alternately for threesuccessive frames and two successive frames so as to convert the firstvideo signal having the first frame frequency into the first videosignal having a second frame frequency, the first frame frequency andthe second frame frequency having a relationship of 2 to 5; outputtingthe second video signal having the first frame frequency alternately forthree successive frames and two successive frames so as to convert thesecond video signal having the first frame frequency into the secondvideo signal having the second frame, the first frame frequency and thesecond frame frequency having a relationship of 2 to 5; and performingcontrol such that timing at which a field group based on fields of thethree successive frames of the first video signal having the secondframe signal is changed to a field group based on fields of the twosuccessive frames thereof matches timing at which a field group based onfields of the three successive frames of the second video signal havingthe second frame signal is changed to a field group based on fields ofthe two successive frames thereof.